Nonbinary Spatially-Coupled LDPC Codes on the 

Binary Erasure Channel 



Amina Piemontese^, Alexandre Graell i Amai^, and Giulio Colavolpe^ 
fUniversita di Parma, Dipartimento di Ingegneria dellTnformazione, Viale G. P. Usberti 181/A, Parma, Italy 
I Department of Signals and Systems, Chalmers University of Technology, Gothenburg, Sweden. 



o 

-i— > 
O 

O 



CO 



(N 
> 
(N 

in 

ON 

o 

(N 



X 



Abstract — We analyze the asymptotic performance of non- 
binary spatially-coupled low-density parity-check (SC-LDPC) 
codes built on the general linear group, when the transmission 
takes place over the binary erasure channel. We propose an 
efficient method to derive an upper bound to the maximum a pos- 
teriori probability (MAP) threshold for nonbinary LDPC codes, 
and observe that the MAP performance of regular LDPC codes 
improves with the alphabet size. We then consider nonbinary 
SC-LDPC codes. We show that the same threshold saturation 
effect experienced by binary SC-LDPC codes occurs for the 
nonbinary codes, hence we conjecture that the BP threshold for 
large termination length approaches the MAP threshold of the 
underlying regular ensemble. 

I. Introduction 

Low-density parity-check (LDPC) codes are a powerful 
class of codes achieving rates very close to capacity for bi- 
nary memoryless symmetric (BMS) channels. Their excellent 
performance, however, usually requires long block lengths 
over the binary field. For short-to-moderate block lengths, 
nonbinary LDPC codes have been shown to outperform their 
binary counterparts JT]. For this reason, nonbinary LDPC 
codes designed over Galois fields of order 2™ (GF™), where m 
is the number of bits per symbol, have received a considerable 
interest in the last few years. Their performance under iterative 
decoding was analyzed in J2), 0. In |2] the density evolu- 
tion (DE) for nonbinary LDPC code ensembles defined with 
respect to the general linear group over the binary field was 
derived for the binary erasure channel (BEC). It was shown 
that the messages exchanged in the belief propagation (BP) 
decoder can be interpreted as subspaces of the vector space 
GF™, which need to be enumerated. Also, it was observed 
in J2| that the BP threshold of some nonbinary LDPC code 
ensembles improves up to a certain m and then worsens for 
increasing values of m. Upper bounds to the maximum a 
posteriori probability (MAP) thresholds of nonbinary LDPC 
code ensembles were also given, and they were conjectured to 
be tight. Later, in |4), the Maxwell construction of (5), relating 
the performance of the MAP and the BP decoder, was shown 
to hold for nonbinary LDPC codes over GFg. 

Spatially-coupled LDPC (SC-LDPC) codes have re- 
ceived notable attention in the recent years due to their 
outstanding performance for a myriad of channels and commu- 
nication problems. For the BEC, it was proved in Q that the 
BP decoding threshold of a binary SC-LDPC code achieves the 
optimal MAP threshold of the underlying LDPC block code 
ensemble, a phenomenon known as threshold saturation. This 



result has been recently extended to BMS channels (8), and the 
same phenomenon has been observed for many other channels, 
such as the multiple access and the relay channel. However, 
despite their excellent performance for long blocks, SC-LDPC 
codes perform poorly for the short-to-moderate block length 
regime, even worse than, e.g., irregular LDPC codes. 

In this paper, we consider nonbinary SC-LDPC codes for 
transmission over the BEC. To the best of our knowledge, only 
the recently submitted paper Q addresses the construction of 
nonbinary SC-LDPC codes and reports some BP thresholds for 
the BEC. However, no analysis on the MAP threshold nor on 
the threshold saturation phenomenon of nonbinary SC-LDPC 
codes is performed in 0. Here, to analyze the asymptotic 
performance of nonbinary SC-LDPC codes of nonbinary SC- 
LDPC codes we first to consider nonbinary LDPC codes 
and, in particular, analyze their MAP threshold. We give 
a systematic and elegant way to generate all subspaces of 
GF™ of a certain dimension, which allows us to compute 
the BP extrinsic information transfer (BP EXIT) curve for an 
arbitrary m and subsequently an upper bound to the MAP 
threshold for nonbinary LDPC codes. We show that MAP 
threshold bound of regular ensembles improves with m and 
approaches the Shannon limit. We then analyze nonbinary SC- 
LDPC codes and show that, contrary to regular and irregular 
LDPC codes for which the BP decoding threshold worsens for 
high values of m, the BP threshold of nonbinary SC-LDPC 
codes with large termination length improves with m (for the 
values analyzed) and tends to the Shannon limit. We also 
show the threshold saturation phenomenon for given m, and 
we conjecture that the BP threshold of SC-LDPC codes with 
increasing termination length saturates to the MAP threshold 
of the underlying regular ensemble. 

II. Nonbinary LDPC codes 

We consider transmission over a BEC with erasure proba- 
bility e using nonbinary LDPC codes defined over the general 
linear group J3 ■ The code symbols are elements of the binary 
vector space GF™ of dimension m. The code block length 
is n symbols, and we transmit on the BEC the m-tuples 
representing their binary image. Therefore, we interpret the 
codeword as a binary codeword of length nm. We denote 
by Xi the ith information bit and by yi the corresponding 
channel output, which is a random variable over {0,1,?}, 
where symbol ? denotes an erasure. The channel outputs 
are collected into the vector y = {yi,--- ,y nm }- Also, we 



denote by y^i the vector of the channel outputs when the 
zth sample is omitted. We denote a regular nonbinary LDPC 
code ensemble as Q(d v , d c ,m), where d v is the variable-node 
degree and d c is the check-node degree. Given a code in this 
ensemble, we associate to each edge of the corresponding 
graph a bijective linear mapping / : GF™ — > GF™, chosen 
uniformly at random. The set of mappings is the general linear 
group GL™ over the binary field, which is the set of all m x m 
invertible matrices whose entries take values on {0,1}. The 
design rate r of a code in the ensemble Q(d v , d c ,m) does not 
depend on m and can be expressed as r = 1 — j L . 

In this work, we are interested in the asymptotic average 
performance of the regular ensemble when n — > oo. The 
asymptotic performance of LDPC codes can be analyzed in 
terms of the MAP and BP thresholds. We denote the MAP 
and the BP thresholds by £ MAP and e BP , respectively. In the 
case of transmission over the BEC, £ MAP is the largest channel 
parameter such that the normalized conditional entropy con- 
verges to zero. The evaluation of £ MAP is not an easy task, but 
an upper bound can be obtained by computing the asymptotic 
average BP EXIT curve, which corresponds to running a BP 
decoder on a very large graph until the decoder has reached 
a fixed point. This can be accomplished by means of the DE 
method ffTOl . Given a code G in the ensemble Q, the BP EXIT 
curve at the ith iteration is defined as 
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where x\ is the estimate delivered by the BP decoder at the 
£th iteration. The asymptotic average BP EXIT curve of the 
ensemble is defined as 



h BP (e) = lim lim Eo[hf' e (e)]. 

£— >-oo n^oo 



(i) 



The curve is zero until e = e BP , at which point it jumps to 
a non-zero value and continues smoothly until it reaches one 
at e = 1. An upper bound for the MAP threshold £ MAP can 
then be obtained by searching the unique value in [e BP , 1] such 
that J^Lap h BP (e)de = r. Operationally, we integrate the curve 
h BP (e) starting at e = 1 until the area under the curve is 
equal to the design rate of the code. Since we consider regular 
ensembles whose BP EXIT curves jump at most once, this 
bound is conjectured to be tight [5 j. (For general ensembles, 
a tighter bound can be achieved by using the extended BP 
EXIT curve 0.) 

A. Density evolution of nonbinary LDPC codes 

The messages exchanged in the BP decoding are real vectors 
of length 2™, the ith element of which represents the a 
posteriori probability that the symbol is i. In ||2), it was 
shown that in the case of transmission over the BEC the 
performance does not depend on the transmitted codeword and 
hence without loss of generality the transmission of the all- 
zero codeword can be considered. Under this assumption, the 
messages arising in the BP decoder assume a simplified form. 
In particular, the non-zero entries of a message are all equal 



and the message itself is equivalent to a subspace of GF™. 
The number of different subspaces of dimension k of GF™ is 
given by the Gaussian binomial coefficient, 
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if k = m or k = 0, 



otherwise. 



(2) 



Since the non-zero elements of a message are equal, it is 
sufficient to keep track of the dimension of the messages 12. 
We say that a message has dimension k if it has 2 k non-zero 
elements. If a message coming from a node has dimension k, 
it means that the symbol is known to be one out of 2 fe possible 
symbols or, equivalently, that at that node m — k relations on 
the bits composing the symbol are known. Let us consider 
the three subspaces of dimension one of GF^, Si = {00,01}, 
S 2 = {00, 10} and S 3 = {00, 11}. Subspaces Si and S 2 are 
representative of the case where one bit has been recovered 
and the other is still erased, while S3 represents the case where 
the two bits are erased but their sum modulo-2 is known. 

Let pjf\k,d c ) be the probability that a randomly cho- 
sen message computed by a check node and directed to a 
connected variable node at the ith iteration has dimension 
k, and let pj e \k,d v ) be the probability that a randomly 
chosen message computed by a variable node and directed 
to a check node at the ith iteration has dimension k. At 
the check nodes, the BP decoder computes the sum of the 
subspaces corresponding to the incoming messages. We have 
the following recursion for d c ^ 3 and c = 4, ■ • • ,d c 13 
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C™. fe P v W(i,d v )pW(i,d v ) 



pW(M = E E c™ jtk pW(i,c-i)pM(j,d v ),, 



i—0 j—k—i 



where CJ 1 



n(fc-i)(fc-3) 
k — i z 



is the probability 



of choosing a subspace of dimension j whose sum with a 
subspace of dimension i has dimension k. 

At variable nodes, the decoder computes the intersection 
of the subspaces corresponding to the incoming messages. 
We denote by P e (i) the probability that the message coming 
from the channel has dimension i, which is equivalent to the 
probability that i bits are erased by the channel, and we have 



P e (i)= ^ . \s*{l-e) m -\ i = 0,.-.,m. 
We have the following recursion for d v ^ 2 and v - 

3,» - ,d v m 

m m— i+fe 

P r i} (k,2)=Y: e v™ k ps)p^{j,d c ) 

i—k j—k 
m m—i-\-k 

Pi^)(M) = E E V^ k P^(i,v-l)pW(j,d c ): 

i—k j—k 



where VI' 



2 (i-k)(j-k) 



is the probability of 



choosing a subspace of dimension j whose intersection with 
a subspace of dimension i has dimension k. 

The asymptotic BP threshold is the largest channel param- 
eter such that the decoding is successful and can be found 

as 

e BP = sup{e e [0, 1] : P^(0,d v ) ^> 1} . 

III. BP-EXIT CURVE AND MAP THRESHOLD 

To draw the asymptotic average BP EXIT curve from (Q]i we 
need to compute the bit erasure probabilities of the extrinsic 
BP decoder P{x\ =?|y~i). In this section, starting from the 
DE equations in the previous section, we obtain the expression 
of the extrinsic messages delivered by the BP decoder. We 
then propose a method to compute the extrinsic bit probability 



P(xf 



i =? i. 



to draw the BP EXIT curve for arbitrary m. 



We can obtain the extrinsic symbol estimate of the BP de- 
coder \&ex t at the £th iteration taking into account all incoming 
messages to a variable node from the connected check nodes. 
Notice that due to the extrinsic nature of the message, the 
channel observations do not contribute in its computation. 
We define P^t (^)> k = 0, • • • , m, the probability that the 



message 



has dimension k and we have the following 



recursion for d v 2 and v = 3, • • ■ , d v 

m m—i+k 

p e ( ff ) (fe>2)=x: e v- k p^^d c )p^(j, dc ) 

i—k j—k 
m m—i-\-k 

piit 1 \k,v) = Y: E v^pgfH^-i^HM. 

i—k j—k 

Finally, we have 

-fext \k) = -fext \k,d v ) . 

To compute (Q]), we let the number of decoder iterations go to 
infinity. We define the following asymptotic quantities 

P ext (fc)= Um P^Uk) 

£— >oo 

Xi = lim x^-p , * ext = lim 

and obtain 



P{xi =?|y~i) = E p (^ =?|dim(*ext) = k,y^)P ext (k) . 

k=Q 

(3) 

Thanks to the tree assumptions, the probabilities in (0 
do not depend on i and (0 is actually the BP extrin- 
sic entropy of a bit h BF (e). To evaluate the probabilities 

P(ii =?|dim(^I' ex t) = k, y^i), we have to enumerate the 
subspaces associated to the message ^ext, so that we can 
compute 



P(x z =?|dim(* ext ) = fc,y^) = 



i 1 ■ 
^E p i fe) (^= ? iy-)' 

"Yl ^ * 



where Pz k \xi =?|y^;) is the extrinsic bit erasure probability 
associated to the zth subspace of dimension k, given an 
arbitrary but fixed ordering. 

We propose an efficient method to identify the subspaces 
and to derive the corresponding extrinsic erasure probability. 
Note that each subspace of dimension k can be interpreted as 
the set of 2 fe codewords of an (m, k) binary linear block code 
of length m. Therefore, we can associate to each subspace 
of dimension k an (m — k) x m matrix (the parity-check 
matrix of the code) such that the symbols corresponding to 
non-zero entries in the message belong to the nullspace of the 
matrix, meaning that the subspace associated to the message 
is the nullspace of the matrix. Thus, finding all subspaces of a 
certain dimension k reduces to find all parity-check matrices 
that generate a different (m, k) code. An efficient way to find 
these matrices is to find all (m — k) x m binary matrices in 
row-reduced echelon form containing no zero rows. The set 
of these matrices is denoted here as lZ m ,k- A row-reduced 
echelon binary matrix is defined as a matrix in which (i) the 
first one in every row is in a column where all other elements 
are zero and (ii) the number of leading zeros increases in every 
row. The nature of these matrices ensures that their nullspaces 
are distinct. Thus, the number of matrices in H. m ,k corresponds 
to the number of different subspaces (or codes) of dimension 
k of GF™. These matrices can be efficiently found by using a 
modified version of the algorithm described in ifTTl . 

Example 1: There are seven different subspaces of dimen- 
sion one of GFj, and we associate to them the matrices in 

^3,1 
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The next step consists of the computation of the era- 
sure probability associated to each subspace. We know that 
P(ii =?|dim( , I' ext ) = 0,y^i) = since if the subspace has 
dimension the symbol is perfectly recovered after BP decod- 
ing. Also, P(ii =?|dim(<I' ext ) = m,y^i) = 1 since in this 
case we have complete uncertainty on the transmitted symbol. 
The cases k = 1, ■ ■ • ,m — 1 are less intuitive. However, 
this problem is easy to solve by interpreting each subspace 
as an (m, k) code with corresponding parity-check matrix 
in lZ nit k and computing the probability of erasure assuming 
transmission over the BEC with erasure probability e. Since 
the length of these codes is generally very short, we can 
evaluate the erasure probability by using the full complexity 
decoding algorithm. Given the zth subspace of dimension k, 
whose nullspace is generated by the matrix R z , we compute 
P^ k) (x, t =?|y~i) as QUI 

£ ^(l-er-iH^l+rank^-rank^)), 

£C[m]\{i} 

where £ C [m] = { 1 , • • ■ m} denotes the index set of erasures 
and i?| denotes the submatrix of R z indexed by the elements 



TABLE I 

Expression of h BP (e) for m = !,■■■ ,4. G m ^ k is the Gaussian binomial coefficient. 



h Bf (e) 

Pext(l) 

(1 + +P ext (2) 
(l + 2e + £ 2 ) 
(1 + 3e + 3e 2 



G 3,l 

+ e 3 ) 



T7T 



(3- 

.'D 



4e - e 2 ) 



+ Pext(3) 



G 3,2 

+ (7 + 18s + 9e 2 - 6e 3 ) 



G " 1 . : 



+ (7 + 12s - 6e 2 +e 3 ) 



TABLE II 

BP AND MAP THRESHOLDS FOR ENSEMBLES G(3, 6,m). 



m 


£bp 


smap 


1 


0.42944 


0.48815 


2 


0.42347 


0.49487 


3 


0.41220 


0.49791 
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0.39890 


0.49920 
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0.38547 


0.49970 
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0.37288 


0.499895 
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0.36154 


0.499965 
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Fig. 1, BP EXIT curves for regular ensembles Q(3, 6, m), with m = 
• ,7. 

of £. In Table U we report the expression of h BF (e) for m = 
1) • ' ' > 4. 

A. Results 

In Fig. [1] we plot the asymptotic average BP EXIT curves 
for the regular ensembles Q(3, 6, to), with to = 1, ••• ,7. 
We recall that the BP threshold £bp is the point at which 
the curve jumps to a non-zero value. The figure shows that 
the BP threshold decreases as the alphabet size increases. 
This is in line with previous results in the literature, which 
state that the performance of the BP decoder degrades by 
moving to nonbinary alphabets when the variable node degree 
distribution is greater than two 13. 

On the other hand, the MAP performance of the considered 
ensembles seems to improve when to increases. In Table [HI 
we list the BP thresholds and the upper bounds on the MAP 
threshold £~map- Note that the £~map rapidly increases with 
to and approaches the channel capacity. Similar results were 
obtained for other ensembles. 

IV. Nonbinary Spatially- coupled LDPC Codes 

We consider nonbinary SC-LDPC code ensembles similar 
to the ensembles defined in l!T2l for binary codes, which are 




Fig. 2. Chains of nine protographs for the regular ensemble 5(3, 6, m). Top: 
non-interacting protographs. Bottom: a coupled chain of protographs. 

derived from regular convolutional protographs by termination. 
Consider as an example the coupling of regular codes with 
cb, = 3 and d c = 6. The protograph for a Q(3, 6, to) ensemble 
is composed of two variable nodes and one check node, 
as shown in Fig. [2 at the top. We consider a chain of L 
protographs, adding a spatial dimension to our code. Since 
these graphs do not interact, the chain behaves like the original 
(3, 6) regular code. An SC-LDPC code is then obtained by 
coupling the protographs: We connect each protograph to one 
neighboring protograph on the left, and to one neighboring 
protograph on the right, as shown in Fig. [2 at the bottom. We 
denote this coupled ensemble as Qc{3, 6, to, L). Locally, the 
connectivity does not change with respect to the underlying 
ensemble. The only difference is at the boundaries, where 
one check node is added on each side to terminate the chain. 
Check nodes at positions i G [1, L — 2] have degree six, while 
the degree of the remaining check nodes decreases linearly 
according to their position. Having lower degree check nodes 
helps the decoder, at the expense of a loss in terms of design 
rate, which is reduced to rs t e(L) = 5 — 7; [7]. By increasing 
L, the rate loss is reduced, while the beneficial effect of low- 
degree check nodes on the BP performance does not vanish. 

To draw the BP EXIT curve for the coupled ensemble, we 
apply the DE described in Section III-AI to each section i G 
[0, L — 1], taking into account of the spatial structure of the 
code. We then compute the extrinsic symbol estimate for each 
section and the corresponding BP extrinsic bit entropy. The 
BP EXIT curve of the SC ensemble is finally obtained by 
averaging over the L entropies of the chain. 

In Fig. [3] we show the BP EXIT curves of the ensembles 
<?c(3, 6, to, L) for to = 1 and to = 3, for several values of L. 
The corresponding BP and MAP thresholds for m = 3 are 
given in Table [Hi] where we also report the Shannon limit 
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Fig. 3. BP EXIT curves for SC ensembles Gc(3, 6, m, L). Compar- 
ison between binary (m = 1) and nonbinary (m = 3) codes, for 
L = 3,5, 9, 17, 33, 65, 129 and 257. 

TABLE in 

Asymptotic thresholds for ensembles 5c(3, 6, 3, L) in Fig. [3] 



EBP 



EMAP 



0.69913 
0.57947 
0.51077 
0.49795 
0.49791 
0.49791 
129 0.49791 
257 0.49791 



0.82738 
0.68328 
0.59026 
0.54169 
0.51847 
0.50813 
0.50272 
0.50065 



0.83333 

0.7 
0.61111 
0.55882 
0.53030 
0.51538 
0.50775 
0.50389 



Esh = 1 — r3,s(L). These results show that the same threshold 
saturation effect observed for binary SC-LDPC codes occurs 
for the nonbinary codes. In fact, the BP threshold for large 
L approaches the MAP threshold of the regular ensemble 

(3, 6,3). Furthermore, since the MAP performance of the 
underlying ensemble for m = 3 outperforms that of the binary 
one (see Table |Hj, the BP thresholds of the nonbinary SC- 
LDPC code saturate to a better value. 

In Fig. 3] we report the BP EXIT curves of the ensembles 
Gc(3i 6, m, 257), for m = 1, ••■ ,7. The figure shows the 
interesting result that, contrary to non-coupled ensembles, the 
performance of SC-LDPC codes under BP decoding improves 
with m and approaches the Shannon limit. Although not 
reported here due to lack of space, we obtained similar results 
for other ensembles. 

V. Conclusions 

We considered nonbinary SC-LDPC codes on the BEC. 
We proposed an efficient method to compute the BP EXIT 
curve for nonbinary LDPC codes and arbitrary alphabet size 
and subsequently an upper bound to the MAP threshold. 
Our analysis showed that performance under MAP decoding 
improves as the alphabet size increases. Furthermore, we 
analyzed nonbinary SC-LDPC code ensembles, showing that 
the same threshold saturation effect observed for binary SC- 
LDPC codes occurs for nonbinary codes. Interestingly, the 
performance of nonbinary SC-LDPC codes under BP decoding 
improves with m and approaches the Shannon limit, contrary 
to the case of nonbinary, non-coupled, LDPC codes. For 




1 

0.9 
0.8 
0.7 
0.6 

Oh 

«a 0.5 
0.4 
0.3 
0.2 
0.1 


0.4 0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6 
e 

Fig. 4. BP EXIT curves for SC ensembles Go (3, 6, m, 257), for m = 
1, ■ ■ ■ ,7. 

instance, for the 5c (3, Q,m, L) ensemble, the BP threshold 
is improved from 0.4881 for the binary case to 0.4997 for 
m = 5 when L tends to infinity. 

Future work includes a formal proof of the threshold sat- 
uration phenomenon for nonbinary SC-LDPC codes, and a 
finite-length analysis. 
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